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Identifying Ancillary Infomiation 
Associated With An Audio/Video Proaram 

Backuround 

The invention relates to communicating ancillary information associated with a 
plurality of audio/video programs, such as television content associated w^ith a plurality of 
channels. 

5 Ancillary information, such as program sub-titles, emergency messages, closed 

caption messages, and program guide information, may be transmitted with regular television 
content. Other types of ancillary information that may be sent with television content include 
enhancement data such as web pages, multimedia information, or other digital data files. 
Ancillary information may be sent in the vertical blanking interval (VBI) of an analog 

1.0 television broadcast signal. Alternatively, the ancillary information may be sent with digital 
television content over a digital transport medium. 

Various standards exist that provide for transmission of ancillary information with 
television content. One standard is the Advanced Television Enhancement Forum (ATVEF) 
Specification, Draft Version l.lr26, dated February 2, 1999. The ATVEF Specification is 

15 designed to provide for transmission of enhancement data along with television content in 
both analog and digital systems, such as cable systems, satellite systems, terrestrial systems, 
and so forth. The combination of the enhancement data and the television content may be 
referred to as enhanced television content. Enhanced television content provides more 
information and options to viewers. For example, a viewer may be presented with the option 

20 of viewing advertisements, educational information, and so forth, while watching regular 
television programming. 

The transmission of ancillary information may be signified to the user by displaying 
an icon indicating that enhanced information accompanies the programming currently 
displayed. This alerts the user to the possibility that additional information is available but 

25 provides nothing useful to help the user decide whether or not the user wishes to review the 
information. 

Thus there is a continuing need for better ways to give the user more information 
about the ancillary information that may have been transmitted with television content. 
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Summary 

In accordance with one aspect, a method may include transmitting video data. 
Ancillary information associated with the video data is also transmitted. Information may be 
transmitted that may be used to identify the content included in the ancillary information. 
5 Other features and embodiments will become apparent from the following description 

and from the claims. 

Brief Description of the Drawings 

Fig. 1 is a block diagram of one embodiment of an information delivery system in 
accordance with the present invention; 
10 Fig. 2 is a block diagram of a transport operator system, receiving system, and server 

according to one embodiment of the information delivery system of Fig. 1; 

Fig, 3 is a view of a screen display in accordance with one embodiment of the present 
invention; 

Fig. 4 is a flow chart showing software resident on the content creator in accordance 
15 with one embodiment of the present invention; and 

Fig. 5 is a flow chart showing software resident on a receiver in accordance with one 
embodiment of the present invention. 

Detailed Description 

Referring to Fig. 1 , an information delivery system 1 0 according to one embodiment 
20 of the invention includes a content creator 12, a transport operator system 14, and a plurahty 
of receivers 16. The receivers 16 may be located at various receiving sites, including homes, 
offices, entertainment facilities, or other locations. The content creator 12 originates 
enhancement data (or other type of ancillary information) and television content (or other 
type of content including audio and/or video data) to be transmitted by the transport operator 
25 system 14. Alternatively, the content creator 12 may create enhancement data with television 
content provided by another source to the transport operator system 14. Enhancement data 
may include graphics (e.g., web pages, multimedia information, or other digital data files), 
presentation layout, and synchronization information. The combination of the enhancement 
data and television content is referred to as enhanced television content. 
30 The transport operator system 14 provides an enhanced television content delivery 

infrastructure that may include terrestrial, cable, satellite, or other types of transmission 
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Tacilitics (cither analog or digital). The television content and enhancement data may be 
transmitted over a transport medium 22, which may be a terrestrial, cable, satellite, or other 
type of link, to the receivers 16. The receivers 16 may include televisions, set-top boxes, 
personal computers, or other types of systems adapted to receive television content and 
5 associated enhancement data. 

As used in this description, the term audio/video (A/V) content is intended to include 
any type of audio and/or video data that may be transmitted or distributed to one or more 
receiving sites for presentation to viewers and/or listeners. As used here, A/V content may 
refer to content that may include both an audio and a video portion or one of an audio or 

10 video portion. Further, ancillary information other than enhancement data may be 

transmitted with the A/V content. For example, ancillary information may include program 
sub-titles, emergency messages, closed caption messages, and program guide information. 

The receivers 16 may further be coupled to a secondary link 20 that may be a data 
delivery communications channel such as the Internet, a DOCSIS network (which is an 

15 interface for cable modems), or other communications link (whether uni-directional or bi- 
directional). DOCSIS stands for Data Over Cable Systems Interface Specifications, and is 
described in DOCSIS, Version 1.0, dated March 1998, as provided by the International 
Telecommunication Union (ITU). The secondary link 20 may be coupled to the transport 
operator system 14 and/or to one or more servers 18. According to some embodiments, 

20 portions of enhancement data associated with the A/V content transmitted over the transport 
medium 22 may be communicated over the secondary link 20 from the transport operator 
system 14, the one or more servers 18, or some combination of such systems. 

In an alternate embodiment, another type of secondary data path can be part of the 
A/V transmission itself, but not tightly tied to a particular A/V channel. For instance, in 

25 MPEG-2 based systems such as ATSC (Advanced Television Systems Committee) or DVB 
(Digital Video Broadcasting), transport stream programs correspond to what is commonly 
thought of as TV channels. MPEG stands for Moving Picture Experts Group, and the 
MPEG-2 standard is described in ISO/IEC 13818-1 (MPEG-2 Systems), ISO/IEC 13818-2 
(MPEG-2 Video) and ISO/IEC 13818-3 (MPEG-2 Audio), dated in 1994 and provided by the 

30 International Organization for Standardization (ISO) and the International Electrotechnical 

Commission (lEC). A description of ATSC may be found in "Guide to the Use of the ATSC 
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Digilal Television Standard/' dated October 1995. DVB standards may be available from the 
web site {http://www.etsi.org}. 

These transport stream programs can contain audio, video, and data (e.g., ancillary 
information), with all of them tightly associated with that single transport stream program. 
5 When a user tunes to the transport stream program, the receiving device knows the audio, 
video, and data that are associated because they are marked as all being part of the same 
program. 

One standard for describing transmission of enhancement data with television content 
is the ATVEF specification, with one version described in ATVEF Specification, Draft 

10 Version 1 .lr26, dated February 2, 1999. Enhancement data may be transmitted in a number 
of different ways from the transport operator system 14 to the receivers 16, depending on the 
type of transport medium 22. For example, with an analog transport medium such as the 
National Television System Committee (NTSC) Standard of the Electronics Industries 
Association, portions of the enhancement data may be sent in the vertical blanking interval 

15 (VBI) of the NTSC transmission. A description of NTSC may be found in the book, "Video 
Demystified: A Handbook For the Digital Engineer" by Keith Jack, published by HighText 
Publications (2d Ed. 1996). Other types of transport media (analog or digital) may provide 
different mechanisms of communicating the enhancement data. 

Enhancement data according to the ATVEF Specification may include enhancements 

20 each having the following components: an ATVEF aimouncement, a resource, and a trigger. 
The three components may be transmitted using Internet Protocol (IP) multicast to the 
receivers. An IP multicast standard is described in Request for Comment (RFC) 1301, 
entitled "Multicast Transport Protocol." RFCs may be available at website address 
{http://wvvw.ietf org/rfc.html} . 

25 Generally, an ATVEF announcement indicates that enhancement data is being 

transmitted, a resource includes one or more files that contain the enhancement data, and a 
trigger synchronizes the enhancement data with the TV transmission. An armouncement may 
describe the location of both the resource stream and the trigger stream. For each television 
(TV) channel, one or more enhancements may be offered as choices presented to the user, 

30 who can select which of the enhancements, if any, to view. The ATVEF Specification may 
utilize a one-way transmission protocol (the Unidirectional Hypertext Transfer Protocol or 
UHTTP, described in the ATVEF Specification) to deliver resource data. 
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The announcements, resources, and triggers associated with an A/V channel may be 
delivered at about the same time as, and with the transmission of, the A/V content on that 
channel. Conventionally, for each enhancement, the resource stream may be delivered along 
with the announcement, with the resource stream stored locally in the receiver 16. If a viewer 
5 so desires, the enhancement data can be retrieved at the receiver from local storage for 
viewing. 

To provide for greater flexibility and/or to alleviate bandwidth concerns of the 
transport medium 22, some embodiments of the invention transmit (using IP multicast) 
enhancement data associated with multiple A/V channels (e.g., TV channels) over a link that 

10 is separate from the transport medium used to transmit A/V content. Alternatively, the link 
may be part of the same delivery mechanism as the A/V content but is not associated with 
any A/V channel, e.g., an MPEG-2 transport stream with ancillary information in a data-only 
program separate from the A/V programs. The separate delivery mechanism to deliver the 
A/V content may be a separate transport stream or a separate link 20 such as a general 

15 purpose data link or some other type of communications link. Thus, according to some 
embodiments, enhancement data is separated from the A/V data at the transport operator 
system 14 (or alternatively, at another source), with the A/V content transmitted over the 
transport medium 22 and the enhancement data transmitted over the secondary link 20 (or a 
separate transport stream). 

20 In the ensuing description, reference is made to receiving enhancement data received 

over the secondary link 20; however, it is contemplated that the enhancement data or other 
type of ancillary information may be received over a separate transport stream such as that 
used with MPEG-2 delivery systems. In addition, reference is made to tuning to a specific 
A/V channel (e.g., TV channel) at the receiving end. It is contemplated, however, that the 

25 receiver system can be tuning instead to one of the A/V transport stream programs in an 
MPEG-2 based systems. 

Thus, generally, tuning to an A/V program may include tuning to A/V channels (e.g., 
TV channels), to transport stream programs (e.g., in an MPEG based system), or to other 
separations or segments of A/V content. Also, associating ancillary information with an A/V 

30 program can thus refer to associating ancillary information with an A/V channel, a transport 
stream program, or other A/V separations or segments. 
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Referring lo Fig. 2, Ihc transport operator system 14, receiving system 16 and the 
server 18 in the system 10 of Fig. 1 are illustrated. The transport operator system 14 may 
include a receiving port 102 to receive information from the content creator 12 over a link 24. 
The received enhancement data may be provided to a controller 106 in the transport operator 
5 system 14. A/V content may be received with the enhancement data through port 102 or 
through a separate A/V receive port (not shown). The controller 106 may be run under 
control of a software routine 108 (referred to as a transport routine). The transport routine 
108 may initially be stored in a storage medium 104 and loaded by the controller 106 for 
execution. Instructions and data of the transport routine 108 may also be stored in the storage 

10 medium 104. The controller 106 may create special announcements to be transmitted with 
enhancement data over a separate link (e.g., link 20), The enhancement data and special 
announcements may be stored in a storage medium 113, which may be transmitted over the 
secondary link 20 through a transceiver 112. 

Alternatively, the enhancement data and special announcements may be transmitted 

15 over the transport medium 22 with the A/V content but in a separate transport stream 
program. In the latter embodiment, different parts of the transport routine 108 (or 
alternatively, different routines) may handle transmission of both the A/V content and the 
enhancement data and special announcements. The transceiver 112 may be a telephone 
modem, a cable modem, or any other type of analog or digital transceiver or transmitter, 

20 including a satellite transmitter, adapted to communicate over the secondary link 20. 

Enhancement data may be stored in a storage medium 126 in the server 1 8 in addition 
to, or instead of, the enhancement data stored in the transport operator system 14. The server 
18 further includes a transceiver 124 coupled to the secondary link 20 and a control device 
128. More than one server 18 may be coupled to the secondary link 20 to store additional 

25 enhancement data. Alternatively, a plurality of servers 18 may be coupled to the receivers 16 
over separate links. 

In the receiving system 16, a receiver circuit 1 14 (e.g., a TV tuner card) is adapted to 
receive content over the transport medium 22 and a transceiver 1 16 is adapted to 
communicate over the secondary link 20. The receiving circuit 1 14 may be associated with 
30 an A/V device driver routine 130 that forwards the received A/V content to application 
software adapted to process and present the A/V content in the receiving system. The 
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transceiver 1 16 may be associated with a network device driver 132 to receive enhancement 
data from the link 20. 

In one embodiment, the network device driver 132 may send received data to a 
TCP/IP (Transmission Control Protocol/Internet Protocol) stack 134. TCP is described in 
5 RFC 793, entitled "Transmission Control Protocol/' dated September 1991 . Data fiows 

through the TCP/IP stack 134 to application software, including an enhancement routine 138 
for receiving and processing enhancement data and a special announcement routine (SA 
process) 136 to receive and process special announcements. In an alternative arrangement, 
the SA process 136 may be part of the enhancement application 138. The application 

10 routines, device drivers, and other routines or programs may be executable on a controller 
120. Such routines or programs may be initially stored in a storage medium 118 and loaded 
by the controller 120 for execution. The SA process 136 is capable of associating 
enhancement data received over the secondary link 20 to the currently tuned A/V channel. 
The SA process 136 may then combine the associated enhancement data with the A/V 

15 content of the currently-tuned TV channel for presentation. Alternatively, the enhancement 
data and special announcements may arrive in a separate transport stream program (e.g., such 
as those in MPEG based systems) over the transport medium 22, in which case it may be 
different parts of the SA routine 136 (or different routines) that process receipt of the 
enhancement data and A/V content. 

20 Referring next to Fig. 3, in accordance with one embodiment of the present invention, 

the user may be given information about the ancillary information that may have been 
transmitted with the television content. For example, in accordance with one embodiment of 
the present invention, a chevron-shaped indicator 32 may be displayed on the display 30 of a 
receiver 16 to indicate the transmission of ancillary information in general. An additional 

25 indicator 34 may be provided to give information about the particular type of content 

provided as ancillary information. Thus, the indicator 34 may be indicative of children's 
content. An additional indicator 36 may be provided to indicate that the children's content is 
available in Spanish, Additional indicators may be provided as well. 

The indicators may provide information about the ancillary information, such as 

30 enhancement data, that has been provided with the television content. This gives the user 
greater information about the ancillary information, enabling the user to make an informed 
decision about whether or not to access the ancillary information. In some cases, the icons 
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32, 34, and 36 may be hyperlinks that allow the user lo mouse click on the icon (or use 
another pointing device) and to jump immediately to the ancillary information. 

Software 52 resident on the server 1 8 or content creator 12, for example, may begin 
by receiving content to be transmitted as indicated in block 54 of Fig. 4. Next, as indicated in 
block 56, the content creator receives the ancillary information that is to be transmitted in 
association with the content previously received. Based on the content of the ancillary 
information, an icon locator is developed as indicated in block 58. The icon locator provides 
a pointer to the location of information about a content-identifying icon. The icon may be a 
graphical symbol that indicates the nature of all or part of the content in the ancillary 
information. Finally, the content, the ancillary information, and the icon locator are 
transmitted through the transport operator 14 to the receivers 16 (block 60). 

Fig, 5 illustrates implementing one embodiment of the present invention. Software 
40, resident on a receiver 16, begins by receiving an announcement stream as indicated in 
block 42. The announcement stream is parsed to locate an icon locator, as indicated in block 
44. 

The icon locator may take a variety of forms. In one embodiment of the present 
invention, the icon locator may be a uniform resource locator (URL). The URL may point to 
an Internet web address containing information about a suitable icon that may be displayed to 
provide the user with information about the content contained within the ancillary 
information. Alternatively, the URL may point to a location in the transmitted ancillary 
information that may be utilized to access and then display a suitable icon. 

Alternatively, a local identifier (LID) may be provided. Particularly where the 
information is not necessarily available on an on-demand basis, a local identifier or LID may 
be utilized to provide a name for a resource such as a content-identifying icon. The use of a 
LID supports cross-references within the content to the resource. The LID may be useful in 
creating hyperlinks or embedding one piece of content in another. The LID scheme enables 
content creators to assign unique identifiers to each resource relative to a given name space. 

Thus, the LID may be utilized to access the icon information repeatedly. Once the 
icon has been stored on the receiver and is identified through the LID, it can be called up 
repeatedly and used over and over again. For example, in one embodiment of the present 
invention, the receiver may be loaded with a plurality of content-identifying icon images 
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before those images arc actually needed. The icon code for one of those icons may be sent 
with an announcement stream that identifies the icon already on the receiver. 

A LID or URL may be transmitted as part of a trigger. Triggers are real time events 
delivered for enhanced television programs. A receiver may set its own policy for allowing 
5 users to enable or disable enhanced television content and triggers may be utilized as a signal 
to notify users of enhanced content availabihty. Triggers generally include a URL and may 
optionally also include a human readable name, an expiration date, and script. The expiration 
date in connection with triggers utihzed to announce the arrival of a particular type of content 
may provide for a limited duration of display of the icon. For example, the expiration date 

10 may also cause the icon to disappear and reappear periodically. 

In some embodiments of the present invention, the user may select from among a 
plurality of pre-received icons that identify content that the user is interested in knowing 
about. Other icons may be left unselected. When information of a type corresponding to a 
selected icon is received, the pre-selected icon may be displayed. 

15 In accordance with one embodiment of the present invention, the announcement 

stream may include script that causes a transparent overlay to be produced over the display 
image on the display 30 of a receiver 16. The transparent overlay may include one or more 
icons to identify the content that accompanies the enhanced television content. In one 
embodiment of the invention, a plurality of icons indicating available content may be 

20 displayed altemately. 

Thus, retuming to Fig. 5, after the icon locator has been parsed, the icon information 
may be accessed (block 46) either from the local system or from the Internet as two 
examples. One or more icons are then displayed as indicated in block 48. If an expiration 
time was transmitted with the trigger data, a check at diamond 50 determines whether the 

25 expiration time has occurred. If so, the flow ends. Otherwise, the icon persists. 

While the present invention has been described with respect to a limited number of 
embodiments, those skilled in the art will appreciate numerous modifications and variations 
therefrom. It is intended that the appended claims cover all such modifications and variations 
as fall within the true spirit and scope of this present invention. 
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1 What is claimed is: 

2 1. A method comprising: 

3 transmitting video data; 

4 transmitting ancillary information associated with said video data; and 

5 transmitting information that may be used to identify the content included in 

6 said ancillary information. 

1 2. The method of claim 1 wherein transmitting information indicative of the 

2 content includes providing a universal resource locator identifying a web page. 

1 3. The method of claim 2 wherein transmitting information indicative of the 

2 content includes providing a uniform resource locator to a web page that includes an 

3 appropriate icon indicative of the content included in said ancillary information. 

1 4. The method of claim 1 wherein transmitting information indicative of the 

2 content included in the ancillary information includes transmitting a local identifier in said 

3 ancillary information that points to a location storing a content-identifying icon. 

1 5. The method of claim 1 further including providing information to produce an 

2 icon indicative of the content included in said ancillary information on a display screen. 

1 6. The method of claim 5 further including providing information that causes an 

2 icon to be displayed as an overlay. 

1 7. The method of claim 6 further including transmitting a duration for the display 

2 of said icon in said ancillary information. 

1 8. The method of claim 7 further including determining whether the icon has 

2 been displayed for the duration set with the ancillary information. 



1 9. The method of claim 5 further including providing information to produce a 

2 plurality of icons indicative of content in said ancillary information. 
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1 10. An article comprising a medium for storing instructions that cause a 

2 processor-based system to: 

3 transmit video data; 

4 transmit ancillary information associated with said video data; and 

5 transmit information indicative of the content included in said ancillary 

6 information. 

1 11. The article of claim 10 further storing instructions that cause a processor- 

2 based system to provide a universal resource locator identifying a web page. 

1 12. The article of claim 1 1 further storing instructions that cause a processor- 

2 based system to provide a uniform resource locator to a web page that includes an appropriate 

3 icon indicative of the content included in said ancillary information. 

1 13. The article of claim 10 further storing instructions that cause a processor- 

2 based system to transmit a local identifier in said ancillary information that points to a 

3 location storing a content-identifying icon. 

1 14. An article comprising a medium for storing instructions that cause a 

2 processor-based system to: 

3 receive video data, ancillary information associated with the video data, and 

4 information that may be used to identify the content included in the ancillary information; 

5 and 

6 cause an icon indicative of the content included in said ancillary information 

7 to be displayed. 

1 15. The article of claim 14 further storing instructions that cause a processor- 

2 based system to receive a uniform resource locator identifying a web page. 



11 
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1 16. The article of elaim 15 further storing instructions that cause a processor- 

2 based system to receive a uniform resource locator identifying a web page that includes an 

3 appropriate icon indicative of the content included in the ancillary information. 

1 1 7. The article of claim 14 further storing instructions that cause a processor- 

2 based system to receive a local identifier in the ancillary information that points to a location 

3 storing a content-identifying icon. 

1 18. The article of claim 17 further storing instructions that cause a processor- 

2 based system to store the local identifier and use the local identifier to locate a content- 

3 identifying icon. 

1 19. A processor-based system comprising: 

2 a processor; and 

3 a transmitter coupled to said processor to transmit video data along with 

4 ancillary information associated with the video data together with information that may be 

5 used to identify the content included in said ancillary information. 

1 20. The system of claim 19 wherein said transmitter transmits a uniform resource 

2 locator that may be used to identify the content included in the ancillary information. 
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